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Executive Summary 

The goal of the No Child Left Behind Act (NCLB) was to 
ensure that states set educational standards in core 
academic subjects, and to hold schools accountable for 
ensuring that all students meet these standards. 

Ratified in 2002, NCLB gave schools approximately 
twelve years, until the end of the 2014 school year, to 
demonstrate that 1 00% of students meet their state 
standards, including students with special needs, those 
with limited English proficiency, and traditionally 
academically disadvantaged groups such has high 
poverty students. Furthermore, NCLB requires schools 
to demonstrate adequate yearly progress (AYP) 
towards meeting the 100% proficiency targets in every 
year between 2002 and 2014. 

Given the inflexibility of NCLB's AYP targets and the 
federal line drawn in the sand for 2014, critics have 
expressed concern that states, particularly those with 
high academic proficiency standards 1 , must eventually 
choose between easing their standards to the point that 
even the lowest performing students can meet them, or 
face increasing federal sanctions of schools, including 
loss of funds, forced reallocations of students within 
districts, and eventually, school closures. There is 
some justification for this concern. Many states have 
indeed lowered their proficiency cut scores since 
NLCB’s implementation (Cronin, Dahlin, Kingsbury, 
Adkins, 2007), including states that had previously 
maintained some of the highest proficiency standards in 
the nation (Dahlin and Cronin, 2009). 

Other criticisms have focused on NCLB’s use of 
proficiency rates as the school performance metric, 
since it only holds schools accountable for the 
performance of students that are below their state 
proficiency standards. Students whose performance 
exceeds their state proficiency standards exert no 
influence on school outcomes, and NCLB does not 
require schools to ensure that students performing 
above state proficiency standards make any kind of 
progress at all (Loveless, Farkas, and Duffett, 2008). 

Implicit in these trends are two assumptions. The first 
assumption is that lowering the proficiency cut scores 
negatively impacts student performance and growth. 
The second is that the implied focus of the current 
accountability system on nearly- or non-proficient (i.e., 



1 For our purposes, when referring to a standard 
or its difficulty, we refer to the proficiency cut 
score on the state assessment. 



“bubble”) students has negative consequences for 
higher performing, already-proficient ones. 

The Kingsbury Center at NWEA is home to one of the 
nation’s largest repositories of information about 
student academic growth, so we examined these two 
assumptions using growth data collected from hundreds 
of thousands of students across the country, a small 
sample from the millions of student records hosted 
within the Kingsbury Center’s Growth Research 
Database. Specifically, we examined two questions: 

1 ) After accounting for differences attributable to 
poverty, race, gender, amount of instruction 
received, and pertinent level school factors, 
does the difficulty of a state’s proficiency 
standards bear any relationship to student 
academic growth? 

2) Do students that are above their state’s 
proficiency standard demonstrate less 
growth, relative to their peers, than do 
students performing below the level of their 
state proficiency standards? 

We investigated these questions separately for four 
samples of roughly 100,000 students, one sample each 
for third and eighth grade students, and for reading and 
mathematics. Samples included growth information 
from students living in 32 states. All students in each 
sample participated in NWEA’s Measures of Academic 
Progress (MAP) test during the fall 2008 and spring 
2009 terms, a computerized-adaptive test that reports 
student performance on an equal-interval cross-grade 
scale. Student growth was calculated by finding the 
difference between the fall and spring scores. 

To create a comparison group for each sample, every 
member of the study group was matched with up to 51 
students who formed a Virtual Comparison Group (VCG 
for that student. Each student in the VCG closely 
matched the study group student on the variables of 
race, gender, starting achievement, the amount of 
instructional time, and a number of pertinent school 
characteristics, including school poverty rate. To 
compare the groups, the growth of each student in the 
study group was compared to the mean growth of the 
student’s Virtual Comparison Group and aggregated. 
The difference, which is expressed as a VCG Growth 
Index expresses the net growth of the study group 
students relative to the growth of their collective VCGs. 

To evaluate the research questions, a FHierarchical 
Linear Model was used. The model evaluates two 




VCG Growth Index 
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independent variables relative to each student in the 
sample, these being the difficulty of the proficiency cut 
score and the status of the student relative to that cut 
score (i.e., whether the student’s fall performance was 
projected to be above or below the proficiency 
standard). Because student performance varies 
across states, the model controls for the student’s state 
as a random effect. Differences in student 
demographic characteristics were not directly controlled 
in the model, but were controlled through the VCG 
matching process. The VCG Growth Index is the 
dependent variable in the model and is used to 
evaluate whatever influence the difficulty of the 
standard and the status of the student relative to it 
might have on growth. 

Although our analyses examined each combination of 
grade and subject separately, our findings were 
consistent across elementary and middle school 
students, and across academic disciplines, with only 
one notable exception as described below. 



Question 1 : Do students who fall below the 
performance level of their state’s proficiency 
standards demonstrate greater growth, relative to 
their peers, than students who are already 
proficient? 

For this question, the model revealed a consistent 
pattern across all grades and subject areas. Students 
who already meet or exceed their state’s proficiency 
standards showed less academic growth over the 2008- 
09 school year, relative to their peers, than students 
below their state's proficiency standards. The 
differences for each grade/subject combination are 
summarized in Figure 1. 

The results indicate that students who meet/exceed 
their state's proficiency standards at the beginning of 
the school year prior to receiving any instruction 
(roughly one half of each of our samples), receive less 
benefit from their instruction than do their lower 
performing counterparts. On average, the differences 
in the VCG growth index for the proficient and not-yet- 
proficient students were between 0.4 scale score points 
and 0.8 scale score points. These differences, while 
not extraordinarily large, are statistically significant and 
would typically represent between about two weeks and 
four weeks of instruction. 



Figure 1. VCG Growth Index for Proficient/Not Yet Proficient Students 




■ Not Yet Proficient 

■ Proficient 



Grade 3 Math Grade 8 Math 



Grade 3 
Reading 



Grade 8 
Reading 




VCG Growth Index 



Kingsbury Center at NWEA 



4 



Question 2: Do students in states with difficult 
proficiency standards show more academic growth 
than students from states with lower standards? 

Differences across states with respect to school 
curricula, geography, poverty levels, and other factors 
all have an impact on academic growth. But our 
analytical approach (hierarchical linear modeling) 
permitted us to tease out differences that might be 
attributable to the difficulty of the state proficiency 
standard and to examine the potential impact from high 
or low state standards separately. For three of the four 
samples, we found no significant relationship between 
the difficulty of the proficiency cut score and the growth 
of our study group relative to their VCG. We did find 
that 3 rd grade math students showed a modest increase 
in academic growth related to the difficulty of state math 
standards (see Figure 2). In other words, setting higher 
third grade math proficiency standards was associated 
with greater growth in math skills. But this pattern was 
not found in the middle school grades evaluated (Figure 
3). The effect of the proficiency cut score is modest; a 
ten point increase in the difficulty of the cut score 
(which would increase its difficulty by about one grade 
level) would improve student growth by 0.4 scale score 
points or roughly two weeks of instruction. 



As can be seen in Figures 4 and 5 for grade 3 and 
grade 8, respectively, no relationship at all was seen 
between growth in reading and state standards. In 
other words, students from states with difficult reading 
standards showed neither more nor less growth in 
reading than students from states with easy standards. 

Implications 

Two major criticisms of NCLB were examined and 
justification for one of these concerns was 
substantiated. Across all four samples, we found that a 
student’s status relative to his or her state proficiency 
bar had an effect on growth, and that students below 
the proficiency bar showed greater growth than those 
above it. This tends to validate concerns that NCLB 
may be focusing the energy of educators on “bubble” 
students, or students below the state proficiency cut 
score who might help the school meet its Adequate 
Yearly Progress requirement if they were to become 
proficient during the school year. 



Figure 2. Impact of State Standards on 
VCG Growth Index For Grade 3 Math 
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Figure 3. Impact of State Standards on 
VCG Growth Index For Grade 8 Math 
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However, the prevailing wisdom that lower proficiency 
standards lead to poorer student outcomes was 
observed in only one of four conditions. In the case of 
third grade mathematics, lower state standards did 
indeed predict modestly poorer growth. However, this 
relationship did not persist into middle school, and it 
was not seen at all in reading. In other words, in three 
of the four cases examined, student growth bore no 
relationship to whether states set their academic 
proficiency standards high or low. 

What did impact growth was not whether state 
proficiency standards were high or low, but whether a 
student was above or below that arbitrary proficiency 
line, wherever it was. Students above that line received 
less benefit from their instruction, relative to peers, than 
did students below that line. 

This finding is consistent with the concerns by NCLB 
critics who claim that measuring only whether students 
exceed state proficiency thresholds means that high 
performing students, whose performance is well above 
proficient, won’t impact school accountability outcomes. 
And since these students don’t impact school 
outcomes, growth opportunities for these high 
performing students are missed. 



Alternatives to proficiency rates such as growth metrics 
that require all students to demonstrate improvement, 
may be a more equitable approach to school 
accountability, and may do more to ensure that all kids 
are reaching their fullest potential. 
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Figure 4. impact of State Standards on 
VCG Growth Index For Grade 3 Reading 



Figure 5. Impact of State Standards on 
VCG Growth Index For Grade 8 Reading 
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